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Abstract. In this paper we study a key exchange protocol similar to 
the Difhe-Hellman key exchange protocol, using abelian subgroups of 
the automorphism group of a non-abelian nilpotent group. We also 
generalize group no. 92 of the Hall-Senior table !I6 to an arbitrary prime 
p and show that, for those groups, the group of central automorphisms 
is commutative. We use these for the key exchange we are studying. 
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1. Introduction 

In this paper we generalize the Diffie-Hellman key exchange protocol from 
a cyclic group to a finitely presented non-abelian nilpotent group of class 2. 
Similar efforts were made in |2l[3l[25] to use braid groups, a family of finitely 
presented non-commutative groups [HHO], in key exchange. We also refer 
to Section 3] for a formal description of a key exchange protocol similar 
to our^. Our efforts are not solely directed to construct an efficient and 
fast key exchange protocol. We also try to understand the conjecture, the 
discrete logarithm problem is equivalent to the Diffie-Hellman problem in a 
cyclic group. We develop and study protocols where, at least theoretically, 
non-abelian groups can be used to share a secret or exchange private keys 
between two people over an insecure channel. This development is significant 
because nilpotent or, more specifically, p-groups have nice presentations and 
computation in those groups is fast and easy [41i Chapter 9]. So our work 
can be seen as a nice application of the advanced and developed subject of 
p- groups and computation with p-groups. 

The frequently used public key cryptosystems are slow and use mainly 
number theoretic complexity. The specific cryptographic primitive that we 
have in mind is the discrete logarithm problem, DLP for short. DLP is gen- 
eral enough to be defined in an arbitrary cyclic group as follows: let G = {g) 
be a cyclic group generated by g and let g'^ = h, where n G N. Given g and 
h, DLP is to find n \i2\ Chapter 6]. The security of the discrete logarithm 
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problem depends on the representation of the group. It is trivial in Z„, but 
is much harder (no polynomial time algorithm known) in the multiplicative 
group of a finite field and even harder (no sub-exponential time algorithm 
known) in the group of elliptic curves which are not supersingular [5] . But 
with the invention of sub-exponential algorithms for breaking the discrete 
logarithm problem, like the index calculus and Coppersmith's algorithm, 
multiplicative groups of finite fields are no longer that attractive especially 
the ones of characteristic 2. 

The discrete logarithm problem can be used in many other groups like 
the group of elliptic curves, in which case a cyclic group or a big enough 
cyclic component of an abelian group is used. In this article we propose a 
generalization of DLP or more specifically the Diffie-Hellman key exchange 
protocol in situations where the group has more than one generator, i.e., 
in a finitely presented non-abelian group. Let / be an automorphism of a 
finitely presented group G generated by {ai, a2, . . . , a„}. If one knows the 
action of / on a € G, i.e., /(a), then it is difficult for him to tell the action 
of / on any other 6 € G i.e., f{b). We describe this in detail later under 
the name "the general discrete logarithm problem" . In this paper we work 
with finitely presented groups in terms of generators and relations and do 
not consider any representation of that group. Though that seems to be a 
good idea for future research. 

Now suppose for a moment that G = {g) is a cyclic group and that we 
are given g and g"^ where gcd(n, |G|) = 1. DLP is to find n. Notice that in 
this case the map x i— > rc" is an automorphism. If we conjecture that finding 
the automorphism is finding n then one way to see DLP, in terms of group 
theory, is to find the automorphism from its image on one element. This is 
the central idea that we want to generalize to non-abelian finitely presented 
groups, especially to a family of p-groups of class 2. This explains our choice 
of the name the general discrete logarithm problem. 

To work with a finitely presented group and its automorphisms the fol- 
lowing properties of the group are needed. 

• A consistent and natural representation of the elements in the group. 

• Computation in the group should be fast and easy. 

• The automorphism group should be known and the automorphisms 
should have a nice enough presentation so that images can be com- 
puted quickly. 

We note at this point that for a p-group the first two requirements are 
satisfied [HI Chapter 9]. 
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2. Our Contribution in this article 

The central idea behind this article is to study a generalization of the 
discrete logarithm problem (DLP) that we call the general discrete logarithm 
problem (GDLP). As a cryptographic primitive the concept of GDLP seems 
to be secure (sec Section 4.1). 

To use GDLP we use a Diffie-Hellman like key exchange protocol using 
finitely presented p-groups with an abelian central automorphism group. 
In this case the security depends not only on GDLP but also on GDHP 
(see Section 4.2) which turns out to be insecure in the specific case we are 
studying. 

Section 8 of this paper contains a brief survey of all the group theoretic 
results necessary for a reader to understand the later part of this paper. 
However, a knowledgeable reader might choose to ignore Section 8 altogether 
and come back to it when required. In Section 10 we survey the existing 
literature for groups with abelian automorphism group and show that none 
of them are adequate for the key exchange we are studying. 

We found no groups readily available in the literature, hence we had to 
develop a family of groups Gn{m,p) with abelian central automorphism 
group (Section 10). This is a significant contribution to the theory of finite 
groups because Gn{m,p) is a generalization of group no. 92 of the Hall- 
Senior table. We describe the group of automorphisms for this group and 
further prove that this group is Miller if and only if p = 2. 

We do not claim that the key exchange protocol is secure. Rather, we 
show that the key exchange protocol is insecure for the particular family of 
groups that we picked. Our study raises two important questions which are 
of interest both mathematically as well as cryptographically. 

a: Are there groups different from G„(m,p), with an abelian central 
automorphism group, for which the key exchange protocol is secure? 

b: Does there exist any cryptographic protocol with reductionist secu- 
rity proof, where the security of the protocol depends only on the 
discrete logarithm problem? If one can find such a protocol using 
cyclic groups then that could be generalized using GDLP, and since 
we claim that GDLP is a secure primitive, this will give rise to a 
secure cryptosystem using non-abelian groups. 

3. Some notations and Definitions 

We now describe some of the definitions and notations that will be used 
in this paper. The notations used are standard: 

• G will denote a finite group. Z = Z{G) denotes the center of the 
group G and will be denoted by Z if no confusion can arise. 
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• G' = [G, G] is the commutator subgroup of G. 

• Aut(G) and Autc(G) are the group of automorphisms and the group 
of central automorphisms of G, respectively. 

• is the Frattini subgroup of G, which is the intersection of all 
maximal subgroups of G. 

• We denote the commutator of a, b by [a, b] where [a, b] = a^^b^^ab. 

• The exponent of a p-group G, denoted by exp(G), is the largest 
power of p that is the order of an element in G. 

The following commutator formulas hold for any element a,b and c in any 
group G. 

(a) : = a[a, b] 

(b) : [ab, c] = [a, c]^[b, c] = [a, c] [a, c, b] [b, c] it follows that in a nilpotent 
group of class 2, [ab,c\ = [a,c][b,c] 

(c) : [a, be] = [a, c][a,bY = [a, c] [a, b] [a, b, c] it follows that in a nilpotent 
group of class 2, [a, be] = [a, b] [a, c] 

(d) : [a,b]-^ = [b,a] 

The proof of these formulas follow from direct computation or can be found 
in [23j. 

Definition (Miller Group). A group G is called a Miller group if it has an 
abelian automorphism group, in other words, if Aut{G) is commutative then 
the group G is Miller. 

Definition (Central Automorphisms). Let G be a group, then (p S Aut{G) is 
called a central automorphism if (!){§) G Z(G) for all g G G. Alternately, 
one might say that (f) is a central automorphism if (/){§) = gZ(f)^g where z^^g G 
Z[G) depends on g and (p. If (p is clear from the context then we can simplify 
the notation as (j){g) = gzg. 

Apart from inner automorphisms, central automorphisms are second best 
in terms of nice description. They are very attractive for cryptographic 
purposes, since it is easy to describe the automorphisms and compute the 
image of an arbitrary element. 

Theorem 3.1. The centralizer of the group of inner automorphisms is the 
group of central automorphisms. Moreover a central automorphism fixes the 
commutator elementwise. 

This theorem first appears in [13] which refers to [T7] and |46| . 

Definition (Polycyclic Group). Let G be a group, a finite series of subgroups 
in G 

G = Go>Gi>G2>G3>...>Gn = l 
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is a poly cyclic series if Gi/Gi^i is cyclic and Gj+i is a normal subgroup of 
Gi. Any group with poly cyclic series is a poly cyclic group. 

It is easy to prove that finitely generated nilpotent groups are polycyclic, 
hence any finitely generated p-group is polycyclic. Let ai be an element in Gi 
whose image generates Gj/Gj+i. Then the sequence {ai, 02, . . . , a„} is called 
a polycyclic generating set. It is easy to see that g £ G can be written as g = 
a^^ag^ . . . a"", where aj are integers. If 5 = a^^ag^ . . . a"" where < Ofj < 
rrii, nii = \Gi : then the expression is a collected word. Each element 

g £ G can be expressed by a unique collected word. Computation with these 
collected words is easy and implementable in computer, for more information 
on this topic see [HI Section 9.4] and also [IF, polycyclic package]. 

4. Key Exchange 

We want to follow the Diffie-Hellman Key exchange protocol using a com- 
mutative subgroup of the automorphism group of a finitely presented group 
G. The security of the Diffie-Hellman key exchange protocol in a cyclic 
group rests on the following three factors: 

DLP: The discrete logarithm problem. 
DHP: The Diffie-Hellman problem. 

DDH: The decision Diffie-Hellman problem P [71 [MIHUlliS] . 

We have already described the discrete logarithm problem. The Diffie- 
Hellman problem is the following: let G = (g) be a cyclic group of order 
n. One knows g, g"' and g'', and the problem is to compute 5"^. It is not 
known if DLP is equivalent to DHP. The decision Diffie-Hellman problem 
is more subtle. Suppose that DHP is a hard problem, so it is impossible to 
compute g""^ from g"', g^ and g. But what happens if someone can compute 
or predict 80% of the binary bits of from g'", g^ and g, then the adversary 
will have 80% of the shared secret or the private key; that is most of the 
private key. This is clearly unacceptable. It is often hard to formalize DDH 
in exact mathematical terms ([TJ Section 3]); the best formalism offered is a 
randomness criterion for the bits of the key. In DDH we ask the question, 
given the triple g°',g'' and 5'^ is c = ab mod n? But there is no known link 
between DDH and any mathematically hard problem for the Diffie-Hellman 
key exchange protocol in cyclic groups. 

Clearly, solving the discrete logarithm problem solves the Diffie-Hellman 
problem and solving the Diffie-Hellman problem solves the decision Diffie- 
Hellman problem. 

As is usual, we denote by Alice and Bob, two people trying to set up a 

private key over an insecure channel to communicate securely and Oscar an 
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eavesdropping adversary. In this paper the shared secret or the private key 
is an element of a finitely presented group G. 

4.1. General Discrete Logarithm Problem. Let G = (ai, 02, . . . , a„) 

and / : G — > G be a non- identity automorphism. Suppose one knows /(a) 
and a e G then GDLP is to find f{b) for any b in G. Assuming the word 
problem is easy or presentation of the group is by means of generators, GDLP 
is equivalent to finding /(aj) for all i which in terms gives us a complete 
knowledge of the automorphism. So in other words the cryptographic prim- 
itive GDLP is equivalent to, "finding the automorphism f from the action 
of f on only one element". 

4.2. General DifRe-Hellman Problem. Let 0,^' : G — > G be arbitrary 
automorphisms such that (pip = ip(p, and assume one knows a, (p{a) and ip{a). 
Then GDHP is to find <p{tp{a)). Notice that GDHP is a restricted form of 
GDLP, because in case of GDHP one has to compute (p{ip[a)) for some fixed 
a, not (p{b) for an arbitrary h in G. There is an interesting GDHP attack 
due to Vladimir Shpilrain. To mount this attack one need not find (p but 
finds another automorphism (p' such that (p'lp = ipcp' and <p'{a) = <p{a). Since 
(p{ip{a)) = ip{(p'{a)) = <p'{ip{a)), the knowledge of the (p' breaks the system. 
We will refer to this attack as the Shpilrain's attack. 

We now describe two key exchange protocols and do some cryptanalysis. 
We denote by G a finitely presented group and S an abelian subgroup of 
Aut(G). 

5. Key Exchange Protocol I 

Alice and Bob want to set up a private key. They select a group G and 
an element a G G \ Z{G) over an insecure channel. Then Alice picks a 
random automorphism (pA € S and sends Bob (pAia)- Bob similarly picks 
a random automorphism cpB & S and sends Alice (pB{a)- Both of them 
can now compute = ^b{<Pa{o)) which is their private key for a 

symmetric transmission. 

Step 1: Alice and Bob selects the group G and an clement a G G\Z{G) 
in public. Notice that G and a are public information. 

Step 2: Alice and Bob picks, at random, two automorphisms (pA and 
4>B from S respectively. Notice that (pA and 4>B are private informa- 
tion. 

Step 3: Alice and Bob compute (pAio) and (psia) respectively and ex- 
changes them. Notice that ^^(a) and (Pb{o) arc public information. 

Step 4: Both of them compute <pAi<pB{a)) = <pB {(pA{a)) from their 
private information; which is their private key. 
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5.1. Comments on Key Exchange Protocol I. Though initially it might 
seem that we do not have enough information to know the automorphisms 
(f)A and it turns out that if we are using automorphisms which fix con- 
jugacy classes, like inner automorphisms, then the security of the above 
scheme actually rests on the conjugacy problem. 

Let0^(a) = ax axid (j)B (a) = y~^ay for some a; and y. Then ^/^(^^(a)) 
= {yx)"^ a{yx) . Since a, ^^(fi) and (I)b{o) are known, if the conjugacy 
problem is easy in the group then anyone can find x and y and break the 
system. 

In the above scheme Oscar knows G and a. If the automorphisms are 
central automorphisms, then he also sees ^^(a) = aZff^^^a and (/>B(a) = 
0'Z^B,a- Oscar can compute -z</,^,o and z^^^a- Now if G is a special p-group 
{G' = Z{G) = then Z{G) is fixed elementwise by both (f)A and (t)B- 

Then 

(pA{<pB{a)) = (t)A{aZ^g^a) = aZct>A,aZ4>B,a- (1) 

Oscar knows a and can compute Z(j,^^a and Z(f,g^a and can find the private 
key (t)A{4'B{(^))- In the literature all examples of Miller p-group with odd 
prime p are special and the above key exchange is fatally flawed for those 
groups. 

6. Key Exchange Protocol II 

In this case Alice and Bob want to set up a private key and they set up 
a group G over an insecure channel. Alice chooses a random non-central 
element g and a random automorphism (f)A^ S and sends Bob (j)A{g)- Bob 
picks another automorphism (ps ^ S and computes (pBi<pAig)) and sends 
it back to Alice. Alice, knowing (pA, computes (f)'^^ which gives her <pB{g) 
and picks another random automorphism 4>h G S and computes (pni'pBig)) 
and sends it back to Bob. Bob, knowing (f>B computes which gives him 
(pH{g) which is their private key. Notice that Alice never reveals g in public. 

Step 1: Alice and Bob set up the group G. Notice that G is public 

information. 

Step 2: Alice picks g E G \ Z{G) and a random (pA G S. Then she 

computes (pAig) and sends that to Bob. Notice that g and 4>a are 

private but (f>Aig) is public. 
Step 3: Bob picks (pB G 5" at random and computes (pB {(pAig)) and 

sends that back to Alice. Notice that (pB is private but (pB {(pAig)) 

is public. 

Step 4: Alice computes (p'^^ and then computing (p^^ {(pB {(pA{g))) she 
gets (pB{g)- 
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Step 5: Alice now picks another random automorphism (pH € S and 
computes {(pBig)) and (pnig)- She then sends (pn {'Psig)) to Bob 
but keeps 4>H{g) private. 

Step 6: Similar to Step 4, Bob computes cpnig)- Now both Alice and 
Bob knows (pnig) and it is their common key. 

6.1. Comments on Key Exchange Protocol II. Notice that for central 
automorphisms, (pA and (pB, (pAig) = gZ(f,^,g; since g is not known Oscar 
doesn't know z^p^^g but if G is special {Z{G) = G' = $(G)) then cpsigz^^^g) 
= gZ(f,g^gZ^j^^g from which z,^^,^ can be computed. Now (f)H{(t>B{g)) = 
gZ(f,g^gZ(j,jj^g is a public information; so using z^jj^^g one can compute gz^fi^^^g, 
which is (pnig) and the scheme is broken. As one clearly sees, this attack is 
not possible if the group is not special. 

The reader might have noticed at this point that all the attacks are GDHP. 
So certainly in some groups GDHP is easy, even though GDLP is hard. 

As we know, any automorphism in G can be seen as a restriction of 
an inner automorphism in Hol(G) (see |291 H5] for further details on the 
holomorph of a group). Solving the conjugacy problem in Hol(G) will break 
the key exchange protocols for any automorphism. On the other hand, 
operation in Hol(G) is twisted so it is possible that the conjugacy problem 
in Hol(G) is difficult even though it is easy in G. Since any cyclic group is 
a Miller group, success of the holomorph attack would prove insecurity in 
DLP. Therefore we believe that the holomorph attack will not be successful 
in many cases. Though more work needs to be done on this. 

7. Key Exchange using Braid Groups 

In [25j a similar key exchange protocol was defined, in this section we 
mention some similarities of their approach to ours. We also mention how 
our system generalizes their system which uses braid groups. See also [8]. 

We define braid group as a finitely presented group, though there are fancy 
pictorial ways to look at braids and multiplication of braids. An interested 
reader can look in [HllO]. The braid group Bn with n-strands is defined as: 

Bn = (o-i, . . . , (Tn-i : o-jCTjCJi = (TjaiGj if \i - j\ = 1, o-jcjj = CTjO-j if |i - jj > 2) 

In [25], the authors found two subgroups A and B of the group of inner 
automorphisms of Inn(i?„), such that, \i (p ^ A and ip B, then 
(p{ip{g)) = 'ip{(p{g)) for g G Bn- Then the key exchange proceeds similar to 
the Key Exchange Protocol I above; with the restriction that Alice chooses 
automorphisms from A and Bob chooses automorphisms from B. There is 
also a different approach to key exchange using braid groups as in [21 [3] . 
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In the same spirit as we can develop a key exchange protocol similar 
to the key exchange protocol I, where we take two subgroups A and B in 
Aut(G) such that for (j) £ A and ip e B, (l){tlj{g)) = ip{(l){g)) for ah g e G. 
The use of inner automorphisms is only possible when the conjugacy or the 
generalized conjugacy problem (conjugator search problem) is known to be 
hard. 

There are significant differences in our approach to that of the approach 
in [25]. In [25], the authors choose a group and then try to use that group 
in cryptography. On the other hand, we take the fundamental concept as 
the discrete logarithm problem, generalize it using automorphisms of a non- 
abelian group and then look for groups favorable to us. The fact that the 
central idea in braid group key exchange turns out to be similar to ours is 
encouraging. 

It is intuitively clear at this point that we should start looking for groups 
with abelian automorphism group, i.e.. Miller groups. 

8. Some useful facts from group theory 

The term Miller Group is not that common in the literature. It was 
introduced by Earnley in [llj. Miller was the first to study groups with 
abelian automorphism group in [34j. Cyclic groups are good examples of 
Miller groups. G.A. Miller also proved that no non-cyclic abelian group is 
Miller. 

Charles Hopkins began a list of necessary conditions for a Miller group 
in 1927 [19]. He complained that very little is known about those groups. 
The same is true today. Except for some sporadic examples of groups with 
abelian automorphism groups, there is no sufficient condition known for a 
group to be Miller. 

We now state some known facts about Miller groups which are available 
in the literature and which we shall need later. For proof of these theorems 
which we present in a rapid fire fashion, the reader can look in any standard 
text books, like [23\ I36|. or the references there. 

Proposition 8.1. If G is a non-abelian Miller group, then G is nilpotent 
and of class 2. 

Proof. It follows from the fact that the group of inner automorphisms com- 
mute and G/Z{G) ^ Inn(G). • 

Since a nilpotent group is a direct product of its Sylow p-subgroups Sp, 
and Aut(yl x B) = Aut(^) x Aut(i?) whenever A and B are of relatively 
prime order, it is enough to study Miller p-groups for prime p. 

Proposition 8.2. IfG is ap-group of class 2, then exp{G') = exp{G/Z{G)). 
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Proposition 8.3. In a p- group of class 2, (xy)^ = x"'y"[y, x] 2 . Fur- 
thermore if exp{G') = n is odd, then {xy)"^ = x^y^ . 

By definition, in a Miller group all automorphisms commute. Since central 
automorphisms are the centralizer of the group of inner automorphisms, we 
have proved the following theorem. 

Theorem 8.4. In a Miller group G, all automorphisms are central. 

It follows that to show a group is not Miller, all we have to do is to 
produce a non-central automorphism. 

Proposition 8.5. If the commutator and the center coincide then every 
pair of central automorphisms commute. 

Proof. Let G be a group such that G' = Z{G). Then let cp and ip be 
central automorphisms given by (/>(x) = xzip^x and ip{x) = xz^^x where 
z^,x,z,p,x e G'. Then 

^p{(p{x)) = ipixz^^x) = ipix)z^^x = xz^^xZ^,x = xz^^xZ-,p,x = (t){ip{x)). 

Definition (Purely non-abelian group). A group G is said to he a purely 
non-ahelian group (PN group for short) if whenever G = A x B where A 
and B are subgroups of G with A ahelian, then A = \. Equivalently G has 
no non-trivial ahelian direct factor. 

Let cr : G ^ G be a central automorphism. Then we define a map 
fa : G ^ Z{G) as follows: fa{g) = g~^CT{g). Clearly this map defines a 
homomorphism. The map a 1-^ is clearly a one-one map. Conversely, if 
/ G Hom(G, Z{G)) then we define a map af{g) = gf{g), x € G. Clearly a/ 
is an endomorphism. It is easy to see that 

Ker(c7/) = {xeG: f{x) = x'^}. 

Hence it follows that aj is an automorphism if and only if f{x) 7^ for 
all X G G with x 7^ 1. 

Theorem 8.6. In a purely non-ahelian group G, the correspondence cr ^ fa 
is a one-one map of Autc{G) onto Hom{G, Z{G)) 

Proof. See [Tj. • 

For any / £ Hom(G, Z(G)) there is a map /' G Hom(G/G', Z(G)) since 
f{G') = 1. Furthermore, corresponding to /' € Hom(G/G', Z(G)) there is 
a map f : G ^ Z{G) explained in the following diagram 

G G/G' Z(G) 
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where r] is the natural epimorphism. 

Let G be a p-group of class 2, such that exp(Z(G)) = a, exp(G') = b and 
exp(G/G') = c and let d = min(a,c). Now from the fundamental theorem 
of abelian groups, let 

G/G' = Ai® A2® ...Ar where Ai = {ai) 

Z{G) = Bi® B2® ...Bs where Bi = {hi) 

r, s G N be the direct decomposition of G/G' and Z(G). If the cyclic compo- 
nent Af. = (ofc) has exponent greater or equal to the exponent of Bj = (bj), 
then one can define a homomorphisms / : G/G' Z{G) as follows 



bj where i = k 
1 where i ^ k 



From this discussion it is clear that for / € Hom(G, Z(G)), f{G) generates 
the subgroup 

7^ = {z G Z{G) : \z\ < d = mm{a, c)}. 

Definition (Height). In any abelian p-group A written additively, there is 
a descending sequence of subgroups 

ADpAZD p^A D . . . D D D . . . 

Then x ^ A is of height n if x & p'^A but not in p'^^^A. In other words the 
elements of height n are those that drop out of the chain in the {n + 1)*'' 
inclusion. 

For further information on height see [22j . 
Since for a class 2 group we have 

exp(G/G') > exp(G/Z(G)) = exp(G') 



it follows that c>b. Hence if d = min(a, c) then either d = b or d > b. 

Let height(j;G') > b, then xG' = y^^G' for some y ^ G. Then for any 
F G Hom(G,G'), F{yG')P' = 1 implying xG' G F'\l). Conversely, let 
height(xG') < b. Then from the previous discussion it is clear that there 
is a F' G Hom(G/G',G') such that xG' is not in the kernel, consequently 
there is a F G Hom(G, G') such that x ^ ker(i<'). Combining these two facts 
we see that: 

JC= Pi F-\1) = {x G G : height{xG') > b} 

FeHom(G,G') 



Proposition 8.7. /C C 7^ 
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Proof. In a class 2 group, if x G /C then xG' = G' for some y £ G and 
exp(G/Z) = 6 and G' C hence x G 

Let X E /C, then height(xG') > 6, hence there is a y € G such that 
yP G' = xG' i.e., X = z where z £ G' and y^'' € G' and c> b. We have 

Hence |x[ < mm(p°' , p'^) which imphes that x £ TZ. • 
Proposition 8.8. For a PN group G of class 2, if Autc{G) is abelian then 

Proof. In a PN group, using Theorem 18. 61 and the notation there, two central 
automorphisms a and r commute if and only if fmfr G Hom(G, Z(G)) 
commute. Then for any / G Hom(G, Z(G)) and F G Hom(G, G') we have 
that foF = Fof = l. Since /(G') = 1, clearly F o f{G) = 1 proving that 
7e C /C. • 

Combining the above two propositions, we just proved that in a PN group 
G of class 2, if Autc(G) is abelian then TZ = K. As discussed earlier there 
are two cases d = h and d > b. Adney and Yen proves that: 

Proposition 8.9. If G is a non-abelian p group of class 2, and Autc{G) is 
abelian with d > b, then TZ/G' is cyclic. 

Proof. See [T, Theorem 3]. • 
Theorem 8.10 (Adney and Yen). Let G be a purely non-abelian group of 

n 

class 2, p odd, let G/G' = Y\{xiG'}. Then the group Autc{G) is abelian if 

i=l 

and only if 

(i) 7^ = /C 

(ii) either d = b or d > b and TZ/G' = {xf G'} 

Proof. See [1, Theorem 4]. • 

From the proof of Proposition 8.5 it follows that in a group G with Z{G) < 
G', the central automorphisms commute. 

Theorem 8.11. The group of central automorphisms of a p- group G, where 
p is odd, is ap-group if and only if G has no non-trivial abelian direct factor. 

Proof. See [37\ Theorem B] and its corollary. • 

At this point we concentrate on building a cryptosystem. We note that 
Miller groups in particular have no advantage over groups with abelian cen- 
tral automorphism group. It is hard to construct Miller groups and there 
is no known Miller group for an odd prime, which is not special. So we 
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now turn towards a group G such that Aut(G) is not abehan but Autc(G) is 
abeHan. We propose to use Autc(G') rather than Aut(G) in the key exchange 
protocols described earher. 

9. Signature Scheme based on conjugacy problem 

Assume that we are working with a group G with commuting inner auto- 
morphisms. 

Ahce pubhshes a and (3 where /3 = a~^aa and keeps a a secret. To sign a 
text X £ G she picks an arbitrary element k £ G and computes 7 = kak~^ 
and then computes 6 such that x = {6k){aj)^^ . Now notice that 

xax-^ = {6k){aj)-^a{{6k){a-fy^)-^ 
= {6k)^^^a^^aa'yk^^6^^ 

= 6^^^ a^^ kak^^ a^S^^ Inner automorphisms commute 

= 6a~^ja5^^ 

5{kj3k'^)5-^ 7 = kak-^ a^Sa = k/Sk'^ 

So to sign a message x G G Alice computes 6 as mentioned and sends x, (kS). 
To verify the message one computes L = xax~^ and R = 6kl3{6k)~^ . If 
L = R then the message is authentic otherwise not. 

There is a similar signature scheme in |24) . where they exploit the gap be- 
tween the computational version (conjugacy problem) and the decision ver- 
sion of the conjugacy problem (conjugator search problem) in braid groups. 
We followed the El-Gamal signature scheme closely |42^ Chapter 7]. 

9.1. Comments on the above Signature Scheme. If one can solve con- 
jugacy problem in the group then from the public information a and /3 he 
can find out a and our scheme is broken. Conjugacy problem is known to be 
hard in some groups and hence it seems to be a reasonable assumption at 
this moment. There is another worry: if Alice sends k and 6 separately then 
one can find a from the equation x = {6k)(a'y)~^ , since 7 is computable. 
However, this is circumvented easily by sending the product 6k not 6 and k 
individually and keeping k random. 

10. An interesting family of p-groups 

It is well known that cyclic groups have abelian automorphism groups. 
The first person to give an example of a non-abelian group with an abelian 
automorphism groups is G.A. Miller in [Mj which was generalized by Struik 
in [43]. There are three non-abelian groups with abelian automorphism 
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group in the Hall-Senior table [TB], they are nos. 91, 92 and 99. Miller's 
example is no. 99. In [20], Jamali generalized nos. 91 and 92. His gen- 
eralization of no. 91 is in one direction, it increases the exponent of the 
group. 

Jamali in the same paper generalizes group no. 92 in two directions, the 
size of the exponent and the number of generators. His generalization was 
restrictive in that it works only for the prime 2. There are other examples 
of families of Miller p-groups in the literature, the most notable one is the 
family of p-groups, for an arbitrary prime p, given by Jonah and Konisver in 
|21j . This was generalized to an arbitrary number of generators by Earnley 
in [11] . There are other examples by Martha Morigi in [35] and Heineken and 
Liebeck in [18] . All these examples of Miller groups given in [111 [T8| [211 [35] 
are special groups, i.e., the commutator and the center are the same. For 
special groups the key exchange protocols do not work as noted earlier. So 
there is no Miller p-group, readily available in the literature, for arbitrary 
prime p which can be used right away in construction of the protocol. The 
only other source are groups nos. 91, 92 and 99 in the Hall Senior table [16] 
and their generalizations, notice that these groups are not special but are 2- 
groups. Of the three generalizations, the generalization of no. 92 best fits our 
criterion because it is generalized in two directions, viz. number of generators 
and exponent of the center and moreover it is not special; Z[G) = yl x G' 
where A is a cyclic group. So once we generalize it for arbitrary primes, 
it has "three degrees of freedom", the number of generators, exponent of 
center and the prime; which makes it attractive for cryptographic purposes. 

In the rest of the section we use Jamali's definition in [20j to define a 
family of p-groups for arbitrary prime. So this family is a generalization of 
Jamali's example and assuming transitivity of generalizations, ultimately a 
generalization of group no. 92 in the Hall-Senior table [16]. We study au- 
tomorphisms of this group and show that the group is Miller if and only if 
p = 2, but this family of groups always have an abelian central automor- 
phism group which is fairly large. We then attempt to build a key exchange 
protocol as described earlier using the central automorphisms. We start 
with the definition of the group 

Definition. Let Gn{m,p) be a group generated by n + 1 elements 
{ao,ai,a2, ■ ■ ■ ,an} where p is a prime number and m > 2 and n > 3 are 
integers. The group is defined by the following relations: 




p 1 p 



1 for 3 < i < n, a' 



■n-l 




[ai,ao] 



1, [a„,ao]=ai, [ai_i,ao]=af for 3<i<n. 
[oj, aj] = 1 for I < i < j < n. 
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We state some facts about the group Gnim,p) whose proof is by direct 
computation (see [30| Section 2.9]). 

a: Gnim, p)' the derived subgroup of G„(m, p) is an elementary abehan 
group (ai,af, ...a^) ~Z^-i. 



c: Gn{m,p) is a p-group of class 2. 
d: Gn{m,p) is a PN group. 

Proposition 10.1. Gn{rn,p) is a poly cyclic group and every element of g € 
Gnim^p) can he uniquely expressed in the form g = a^^al^a'^^a'^^ ...a"", 



< aj < p /or i = 0, 1; < a2 < p"", < < p"^ for i = 3,4, . . . ,n. 

Proof Define Go = Gn{m,p) = (ao,ai,a2, • • • ,«n), Gi = (ai, 02, • • • an) and 
similarly Gk = {ok, a/c+i, • • • , On) for k < n. Since Gi is a finitely generated 
abelian group, it is a polycyclic group [4T1, Proposition 3.2]. It is fairly 
straightforward to show that 



is a polycyclic series and {ai, . . . ,a„} a polycyclic generating sequence of 



It is easy to see from the relations of the group that Gi is normal in 
Go and Gq/Gi is cyclic. It is also straightforward to show that (ajGi+i) = 
Gi/Gi+i and lajGj+i | = \ai\ and hence any element of the group has a unique 
representation of the above form. We would call an element represented in 
the above form a collected word. See also |41| , Chapter 9, Proposition 4.1]. • 

Computation with Gn{m,p): Our group Gn(jn,p) is of class 2, i.e., com- 
mutators of weight 3 are identity, computations become real nice and easy. 
Let us demonstrate the product of two collected words g = Oq " a"^ ^ ag ^ a^^" 
and h = aQ°a^^af^ 03^04*. To compute gh we use concatenation and form the 
word al'^ a"^"^ a'^^ a^* Oq'^ a^^ a^^ a^^ and note that Oj's commute except 
for ao hence one tries to move towards the left using the identity 



can be moved anywhere. Once oq is moved to the extreme left the word 
formed is the collected word of gh. This process is often referred to in the 
literature as collection. Computing the inverse of an element can be similarly 
done. 

We now prove that the group of central automorphisms of the group 
Gn{m,p) for an arbitrary prime p is abelian. For sake of simplicity we 



b: Z{Gn{m,p)) = {al) x G' . 



where 



Gi>G2>...>G„>(l) 



Gi. 
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denote Gn{m,p) by G for the rest of the article, and use notations from 
Theorem [lini 

Lemma 10.2. InG,TZ = Z{G) = K. 

Proof. Using the notation from Theorem 18. lUt we see that in G, a = m — 1, 
6=1 and c = m hence d = m — 1. Clearly, TZ = Z{G) hence K, C Z{G). 
Let X G Z{G), if X £ G' then height(xG') = oo and we are done. If not, 
then X = ziZ2 where zi E (ag) and Z2 € G' . Then xG' = ziG' and hence 
height(xG') > 1. • 

It is easy to see that TZ/G' = Z{G)/G' = {a^G') and hence from Theorem 
I8.1UI we prove the following theorem: 

Theorem 10.3. Autc{G) is abelian. 

10.1. Automorphisms of Gn{m,p). In this section we describe the auto- 
morphisms of groups of this kind. The discussion is, in more than one way, 
an adaptation of the work of Jamali [20] and generalizes his main theorem. 

Lemma 10.4. Let x = aQ° a^^ . . . an" , where (3i, i = 0,l,2...,n are 
integers be an element of G. If p = 2 then (3q is 1 and 

. x2 = a{"af'af . . . al^^ alr^^\l{' for p = 2. Where 7^ = 2(A_i + 
A). 

. xP = af'af' . . . afv^a^T^+^^af " for p^2. 

Proof. For the case p = 2 we just collect terms and use the relation a^__^ = 
al 

For p ^ 2 using Proposition 8.3 we have 

xP = {a^,^a^,^a§\..at-i'aty 

= {a^,r{4'4'---4-\'^tr 

— Uq U2 0.3 

Using the relation a?'_i = a^ we have 



*0 "2 "3^^^ ■ ■ ■ "n^" — "2 "3 • • • "71-2 "n-l 



U/fi U/Q tin • • • Uj^ U/Q tin . . . tt„ - " " 



For the group G we note that H = (oi, 02, as, . . . a„) is the maximal abelian 
normal subgroup of G and is characteristic. It follows that the is also 
characteristic. Following [20], we define two decreasing sequences of charac- 
teristic subgroups {Ki}^~i^ such that 

Ko = H and Ki/Kf_^ = Z{G/Kf_^) (1 < i < n - 1) 
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and {Li} such that 

Lq = H and Li 
It follows easily that 



= {h: heH, hP e [G, L,_i]} (1 < i < n - 1) 



Ki = {ai,a2, . . ■ ,an-^,al^_■_^_^, . . . , a^) 1 < i < n - 1 

Li = (ai, 17,03, . . . ,a„) 
Li = {ai,v,al, . . . ,af^_^,ai+2, • • • ,a„) 2 < i < n - 1 

— 1 

where w = . For 3 < i < n we have 

Kn-inLi_2 = (ai,i;,af,...af_^,ai,af^p...a^) = {v,ai,G'). 

Also /s:„_2 n Lo = {a2,G'). 

Since {v,ai,G') and {a2,G') are characteristic, for any € Aut(G), 

61(02) = «2^-z where z G G' and A;2 G N 

e{ai) =a'^'v'''z where z G G'; fcj G N; i = 3, 4, . . . , n; < < p. 

It is clear that not all ^2 and /cj will make 9 an automorphism. To begin 
with, if 9 is an automorphism then gcd{ki,p) = 1 for all ki, and we may 
choose ki, such that < fcj < p for « = 3, 4, . . . , n. 

Let ^(ao) = a^^'a^'a^^.-at- Since 0(ag) = ^(a^_i) = ^(a„_i)P = 
, from Lemma 10.4 

af_"- = af ^af 3 . . . af_r af",-'^^^" a'^^'^ for p / 2 

implying /3o+/3n-i = mod p, p"^^^\P2 andp|/3j for i = 3, 4, . . . ,n-2,n. 

Hence 9{ao) = ag^^n-T^^-^^ where < r < p. We changed /?o to feo to 
maintain uniformity in notations. 

Notice the relation [a,, oq] = a^_^_i for i = 2, 3, . . . , n implies that 

[^(a,),^(ao)] =^(a,+lf = aSf^ 
It follows that [a^"', 4«a(^'ri'] = ajf' which is the same as [af% ag°] = aJ\+\ 
which implies that [oj, ao]'^'''^' = of+i^^- Recall that G is a p-group of class 
2 and commutes with for i > 2. From these we have a recursive 

formula for ki, (also see [301 Theorem 2.9.7]): choose ko such that < A;o < p 
and k2 such that < A;2 < p™' and gcd(fc2)P) = 1 and then define fcj+i = k^ki 
mod j3 for i = 2, 3, 4, . . . , (n — 1); and ki = k^kn mod p. In [20l Proposition 
2.3] Jamali proves that for p = 2, all automorphisms of G are central. We 
have just proved that for p ^ 2 there is a non central automorphism, take 
/cq > 1; the following theorem follows from Theorem 8.4. 

Theorem 10.5. The group Gn{m,p) is Miller if and only if p = 2. 
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10.2. Description of the Central Automorphisms. Notice that G is 
a PN group, so there is a one-one correspondence between Autc(G) and 
Hom(G, Z(G)). Since it is known from our earhcr discussion that Z{G) = 
{al)xG',Rom{G,Z{G)) = Hom(G, (af )) xHom(G, G'). It follows: Autc(G) = 
A X B where 

A = {ae Aute(G) : x'^a{x) G (a^)} 

B = {ae Autc(G) : x-^a{x) G G'} 

Elements of A can be explained in a very nice way. Pick a random integer 
k such that k = Ip + 1 where < I < p™"^ and a random subset R (could 
be empty) of {0, 3, 4, . . . n}, and then an arbitrary automorphism in A is 

fj(ai) = ai 
£7(02) = 02 



a{ai) 



a ieR 

(2) 



We use indexing in {0, 3, 4, . . . , n} to order R and < < p is an integer 
corresponding to i & R. Conversely, any element in A can be described this 
way. It follows from the definition of A that 

\A\ =p™-i X =p™+«-2. 
The automorphism ^ G -B is of the form 

(h( ) = f ^ = C3^ 

\ UiZ if X = tti, i G {0,2,3,... ,n} ^' 

where z G G' . 

G 

We note that is an abclian group and hence Inn(G) is abelian and 

Z{G) 

hence Inn(G) C Autc(G). We further note from the commutator relations 
in G that Inn(G) C B. 

10.3. Using these automorphisms in key-exchange protocol I. Let 

us briefly recall the key-exchange protocol described before. Alice and Bob 
decide on a group G and a non-central element g G G\Z{G) in public. Alice 
then chooses an arbitrary automorphism and sends Bob (t>A{g)- Similarly 
Bob picks an arbitrary automorphism (j)B and sends Alice (psig)- Since the 
automorphisms commute, both of them can compute (j)A{'pB{9))^ which is 
their private key. The most devastating attack on the system is the one in 
which Oscar, looking at g, (pAig) and ^sig), can predict what 4>A{(t>B{g)) 
will look like, i.e., a GDHP attack. 
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Definition (Parity condition for elements in G). If g = aga^^ 02^3 • • • 
is an arbitrary element of G, i.e., < (3o < p, < Pi < p, < (32 < p™" 
and < Pi < p"^ for 3 < i < n. Then the vector v := {Po,P3,Pa, • • • ,Pn) 
is called the parity of g. Two elements g and g' are said to he of the same 
parity condition if v = v' mod p, where v' is the parity of g' . 

Lemma 10.6. If cj) : G ^ G is any central automorphism then g and (l){g) 
have the same parity condition for any g ^ G. 

Proof. Notice that an automorphism (p either belongs to A or 5 or is of the 
form (pig) = gfcj,ig)g<f,{g) where f^ G Hom(G, and g^ G Hom(G, G')- 

So we might safely ignore elements from A, since they only affect the expo- 
nent of 02- Also note that ai being in the commutator remains fixed under 
any central automorphism. 

So we need to be concerned with elements of B, from the description of B, 
and each commutator is a word in p-powers of the generators and from the 
fact that G' C Z{G), the lemma follows. • 

Now let us understand what an element in A does to an element g G G. 
We use notations from Equation [2j 

Lemma 10.7. Let g = a'^'^ a^"" . . . a^"" , (f) £ A and if 

^{g) = aj"af af^af^ ...a^" then A = P[ fori^2 and 
= kP2 +P"'-^ E rA mod p"". 

Proof. Notice that from Equation [21 it is clear that elements of A only 
affect the exponent of 02, so (5^ = (5i for i ^ 2 follows trivially. From 
the definition of A and simple computation it follows that P2 = fc/32 + 

pin-l j^Q^ , 

In the key exchange protocol I, we will only use automorphisms fronj^ A. 
As noted earlier there are two kinds of attack, GDLP (the discrete logarithm 
problem in automorphisms) and GDHP (the Diffie-Hellman problem in au- 
tomorphisms). We have earlier stated that GDLP is equivalent to finding 
the automorphism from the action of the automorphism on one element. 
It seems that for one to find the automorphism discussed in the previous 
lemma, one has to find k, R and r^. Notice that P2 = kP2 -\- p'^~^ ^ rj/3j 

mod p*", is a knapsack in P2 and Solving that knapsack is not enough 

to compute the image of any element, because R is not known so /3j's are not 
known. We shall show in a moment that the security of the key exchange 



In light of Lemma 10.6, we believe that adding automorphisms from B is not going 
to add to the security of the system. 
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protocol depends on the difficulty of this knapsack, but solving this knap- 
sack does not help Oscar to find the automorphism, just partial information 
about the automorphism comes out. 

Next we show that though it seems to be secure under GDLP, but if the 
knapsack is solved then the system is broken by GDHP. This proves that 
GDHP is a weaker problem than GDLP in Gn(m,p). Let g = a^^a^^a^a^^ 
...an", then as discussed before for ^ Autc(G), with notation from 
Equation 2 and /cj G N for i = 3, 4, . . . , n: 

^2/32+?™-! E nfSi 
m=4'4'a, 4^^'^^...ai-+'-^ (4) 

^[g) = a*af a2 . . . a^^+'^" (5) 

From direct computation it follows that the exponent of 02 in (j){'ilj{g)) is 




\ ieR' 

where k2 = Ip + 1 and k2 = I' p + 1, < I , I' < p"^^^. The exponent of ao, oi 
stays the same and the exponent of ai will be /3j + (fcj + k'-)p mod p^ for 
3 < i < n. As mentioned before since we are using only automorphisms 
from A, i.e., (j) and ip are in A hence ki = k'^ = for i = 3, 4, . . . , n. 

Notice that g, Equations [Hand [5] are public, so Oscar sees those. Since the 
exponents of oq, ai, 03, . . . , a„ are predictable, the key Alice and Bob want 
to establish is the exponent of 02 in which is given by Equation 

6. Since Oscar sees Equations H] and O if he can compute k2 from k2f32 + 
pm-i T[^Q(i then he can compute p^~^ ^ ViPi and the scheme 

is broken. But, k2 = lp + 1 for some I € [0,^"^""'^) hence 
fc2/32+J5'"-i J^r.ft modj^™ 

i&R 

reduces to 

/?2 + /p/?2 + P""^ ^^^^ P"^- 
i&R 

Since (52 is public, Oscar can compute lp(52 +p'^^^ Yl "^iPi mod p^. Notice 

that finding /c2 is equivalent to finding /, hence one of the security assump- 
tions is that there is no polynomial time algorithm to find I from 

lpf32+p'^'^yrA modp'". (7) 



Let us write 



M = /p/32+p"-^X]^^^^ modp"^, (8) 
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then 

M = lp(32 modp""'^. 
Now, if Ip < and gcd(/32,p) = 1, then one can find Ip from the above 

equation and the scheme is broken. So the only hope of making a secure 
cryptosystem out of key exchange protocol I and the group Gn{rn,p) is to 
take / = kp"^~'^ where A: = 0, 1, 2, . . . , (p— 1). In this case, if we set I = lp"^~'^ 
and I' = l'p"^~'^ in Equation [6l then the key will be 

(1 + Ip"'-^) ( (1 + /'p'""')/52 "^if^i] E^^/^^ 

V ieR' J ieR 



[l + + /32 + P™-^ E <Pi + P™'^ E nPi mod 



;i + /p™-i) /32 + p™-^ E + /'p'"~^/32 + P*^-^ E ^'A mod P"" 

Now the information in the last equation is easy to compute from the 
public information, Equations S] and [Sj so the Key Exchange Protocol I is 
broken for automorphisms from A of Gri{m^p) when gcd(/32,p) = 1- 

Now if gcd(p, /32) 7^ 1, i.e., /?2 = ^P* for some i £ [l,m) and 1 < < p, 
then an attack similar to the above breaks the system. The insight behind 
these attacks is that any solution to Equation [5] can be thought of as the im- 
age of g under an automorphism (j)' £ A. We are talking about a solution to 
Equation [U which is easy to find, for which ({)'{§) = M and then Shpilrain's 
attack breaks the system. 

11. Implementation 

There is not much reason left to go into the details of an implementation. 
We briefly mention that this cryptosystem can be implemented without any 
reference to the group G„(m,p). Once the element 

fixed, Alice can send Bob k2j32 + p"^~^ E niod p*" and similarly Bob 

ieR 

can send Alice A;2/^2 + p™^^ E ^ift mod p^. Since Alice and Bob know 

ieR' 

their own ^2) E ^2' E ^ift respectively, they can both compute 

ieR ieR' 
the private key or the shared secret. Since the only operation involved in 

computing the private key is multiplication and addition mod p*", there can 

be a very fast implementation of this cryptosystem. 

12. Conclusion 

In this paper we studied a key exchange protocol using commuting auto- 
morphisms in a non-abelian p-group. Since any nilpotent group is a direct 
product of its Sylow subgroups, the study of nilpotent groups can be reduced 
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to the study of groups. We argued that our study is a generahzation of 
the Diffie-Hellman key exchange and is a generahzation of the discrete log 
problem. Other public key systems like the El-Gamal cryptosystem which 
uses the discrete logarithm problem is adaptable to our methods. This is 
the first attempt to generalize the discrete logarithm problem in the way we 
did. 

We should try to find other groups and try our system in terms of GDLP 
and GDHP. As we noted earlier, GDHP is a subproblem of the GDLP, and 
we saw in G'„(m,p), GDHP is a much easier problem than GDLP. Our 
example was of the form d > b in Theorem 18.101 The next step is to look 
at groups where d = b. We note from Theorem 18. IH if a p-group G is a PN 
group then Autc(G) is a p-group and since p-groups have nontrivial centers; 
one can work in that center with our scheme. In this case we would be 
generalizing to arbitrary nilpotentcy class while still working with central 
automorphisms . 

Lastly we note that, if we were using some representation for this finitely 
presented group G, for example, matrix representation of the group over a 
finite field F^; then security of the system in Gn{m,p) becomes the discrete 
logarithm problem in a matrix algebra |321 [33] . Since the discrete logarithm 
problem in matrices is only as secure as the discrete logarithm problem in 
finite fields, there is no known advantage to go for matrix representation, 
but there might be other representations of interest. 

There is one conjecture that comes out of this work and we end with that. 

Conjecture 12.1. If G is a Miller p- group for an odd prime p, then G is 
special. 
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